System, method, and apparatus for detecting a projected point on computer controlled displayed images

ABSTRACT

A system, method, and apparatus are described in which a light point projected on a computer controlled displayed image is detected. A light point is projected at the displayed image at essentially the same time as a notification signal is transmitted indicating that the light point is being projected. At least one of image capture and image display parameters are adjusted to increase the detectability of the light point within a predetermined time interval after receipt of the notification signal. The image of the displayed image and the light point are captured within the predetermined time interval and the captured image data is analyzed to detect the light point dependent on the adjusted parameters.

FIELD OF THE INVENTION

[0001] The present invention relates to a system and method ofinteracting with a computer controlled display system with a projectedlight signal, and in particular, this disclosure provides a system andmethod of detecting the projected light signal on the computercontrolled display which can allow a user to interact with the displaysystem with the projected light signal.

BACKGROUND OF THE INVENTION

[0002] Displayed images (e.g., slides) projected onto a display screenor display area are often used during an oral presentation. Thedisplayed images serve to enhance or supplement the oral portion of thepresentation. Often the image data for driving a display device fordisplaying the images is provided by a computer system. Commonly,specific software applications are run by the computer system (e.g.,slide generation software application) to generate the image data to bedisplayed by the display device.

[0003]FIG. 1 shows an example of a prior art computer controlled displaysystem including a computer system 10 for providing image data 10A fordriving a display device 11 to project an image (designated by dashedlines 11A and 11B) on a display area 12.

[0004] In this type of system, a presenter commonly uses a pointingdevice such as a light or laser pointer 15 to project a light or lasersignal toward the displayed image thereby projecting a single lightpoint onto the displayed images. Typically, the purpose of the projectedpoint is to bring the audience's attention to a particular location onthe displayed image. In addition to the laser pointer 15, the presenteroften uses an input device 14 such as a mouse to control the computersystem generating the image data 10A and more specifically, to controlthe manner in which the image data is displayed. Commonly the inputdevice is a remote mouse that transmits control signals via infraredoptical signals to change from one displayed slide to the next.

[0005] In one known technique, the display system further includes animage capture device 13 that captures the displayed image (designated bydashed lines 13A and 13B) and any object projected on or positioned infront of the displayed image. The captured data 13C is provided to animage analyzer 16 that determines the location of the objects andprovides this location information 16A to the computer system 10.Providing the location information of objects, such as pointing sticks,fingers, or light pointers, to the computer system can provide a userthe ability to interact with the displayed image using these pointingdevices.

[0006] However, detection of an object used for pointing (e.g., a woodenstick or a finger) positioned in front of a displayed image using thesystem shown in FIG. 1 is more reliable than detection of a light pointprojected on the displayed image. Specifically, in the case of detectingan object, a technique is known using the system shown in FIG. 1 thatreliably detects the pixel locations of the object in the captured imagedata by separating foreground image data from the displayed image datawithin the captured image data. However, it is often difficult todetermine whether the detected pixels using this technique correspond toa laser point or if the detected pixels are produced by a noisy systemor environment. The main reason for this is because a laser point tendsto cover only a few pixels locations within the captured image data inthe same way as noise manifests itself. This problem of detecting thelaser point can be further exacerbated by poor lighting conditions,inferior image capture or display devices, or a highly dynamic image.

[0007] In another known technique, the chromatic nature of the laserpointer (i.e., typically a single highly saturated color) is used todetect a laser point within captured image data by detecting pixelvalues have an anticipated specific color and intensity that is unlikethe displayed image data 10A. However, manufacturing variations canaffect the laser pointer intensity and as a result, detection relying onan anticipated laser intensity may be problematic.

[0008] Hence, what would be desirable is a system and method forreliably detecting a light point on a computer controlled displayedimage.

SUMMARY OF THE INVENTION

[0009] A system for reliably detecting a projected light point on acomputer controlled display is described. A display system includes acomputer system for providing image data and controlling the display ofthe image data by a display device. The display system further includesa device for projecting a light point at the displayed image whilesimultaneously transmitting a notification signal indicating the lightpoint is being projected. A controller adjusts at least one of imagecapture or image display parameters to increase the detectability of thelight point for a predetermined time interval after receipt of thenotification signal. An image capture device captures image dataincluding the displayed image and the projected light point during thepredetermined time interval. An analyzer detects the light point withinthe captured image data dependent on the parameter adjustments. In oneembodiment, image display parameters are adjusted by reducing the imageintensity of the displayed image data to at least a value that is lessthan a maximum value and the light point is detected by identifyingvalues greater than the maximum value within the captured image data.

[0010] An apparatus is described where upon activation, the apparatusprojects a visible light signal while simultaneously emitting anotification signal indicating the light signal is being transmitted.

[0011] A method for use in a system for detecting a light pointprojected on computer controlled displayed images is described. Themethod comprises projecting a light point on the displayed image whileat essentially the same time transmitting a notification signalindicating the light point is being projected; adjusting at least one ofimage capture or image display parameters to increase the detectabilityof the light point within a predetermined time interval after receipt ofthe notification signal; within the predetermined time interval,capturing image data including the displayed image and the projectedlight point; and analyzing the captured image data to detect the lightpoint within the captured image data dependent on the adjustedparameters.

BRIEF DESCRIPTION OF THE DRAWINGS

[0012]FIG. 1 shows a prior art system for interacting with a computercontrolled image display system using a separate light projecting deviceand an input device;

[0013]FIG. 2 shows an embodiment of a system for detecting a light pointprojected on computer controlled displayed images in accordance with thepresent invention;

[0014]FIG. 3 shows an embodiment of an apparatus for projecting a lightsignal on a computer controlled display area and transmitting anotification signal indicating the light signal is being projected atessentially the same time;

[0015]FIG. 4 shows a first embodiment of a method for detecting aprojected light point on computer controlled display images inaccordance with the present invention;

[0016]FIG. 5 shows a second embodiment of a method for detecting aprojected light point on computer controlled display images inaccordance with the present invention.

DETAILED DESCRIPTION OF THE INVENTION

[0017] A system and method of the present invention provides a techniqueof detecting a laser point projected on computer controlled displayedimages is described. An apparatus for facilitating the system and methodis also described. It should be noted that for purposes of the subjectdisclosure a computer controlled display system includes at least acomputer, a processing system or device, or a computing system or devicefor generating and controlling the display of image data; a display areafor displaying the images; and a means for displaying the image data(e.g., projector) in the display area which is controlled by thecomputer, processing system or device, or computing system or device.

[0018]FIG. 2 shows an embodiment of the system of the present inventionincluding a computer system 20, a display device 21, and display area22, an image capture device 23, a projection device 24, and a laserpoint detector 25 comprising a controller 25A and an analyzer 25B. Thecomputer system 20 provides image data 20A. In one embodiment, thecomputer system includes at least a storage area (not shown) for storingimage data 20A. In another embodiment, the computer system includes asoftware application (not shown), such as slide presentation generationsoftware, for generating and providing image data 20A. The image data20A drives the display device 21 to display an image (designated bydashed lines 21A and 21B) onto display area 22. The display area may bea display screen, monitor, or may simply be a wall.

[0019] Light projection device 24 projects a light signal 24A, which inturn, causes a light point to be projected onto the displayed image(21A-21B). At essentially the same time, device 24 transmits anotification signal 24B that indicates the light signal is beingprojected. It should be noted that in accordance with the system shownin FIG. 2, the notification signal can be transmitted by one of a wired(solid line) or wireless (dashed line) transmission. In one embodiment,the notification signal can be transmitted to the computer system 20that, in turn, transmits it to controller 25A. Alternatively, thenotification signal can be transmitted directly to laser point detector25 for coupling to controller 25A. The notification signal can betransmitted via an infrared signal. In one embodiment, activation ofdevice 24 to project the light signal and transmit the notificationsignal is achieved by a single control option (e.g., depression of asingle button) on device 24. The device 24 can be a pointing device usedduring a slide presentation by a presenter to identify locations ofimportance on the display area such as a laser pointer. Alternatively,the pointing device can be a wired or wireless computer mouse includinga laser projector.

[0020] The notification signal 24B is received by controller 25Adirectly or from the computer system 20. In response, controller 25Aadjusts either image capture or image display parameters so as toincrease the detectability of the light point within a predeterminedtime interval occurring after the receipt of the notification signal24B. Specifically, controller 25A can transmit control signals on bus 26to either the computer system 20, display device 21 or the image capturedevice 23 to adjust image display and/or image capture parameters in away so as to increase the detectability of the light point within thedisplayed image captured by image capture device 23.

[0021] The image capture device 23 captures an image (designated bydashed lines 23A and 23B) within the predetermined time interval. Thecaptured image data includes the displayed image (21A-21B) and the lightpoint projected by light signal 24A onto the displayed image. Sinceimage capture and/or image display parameters are adjusted during thepredetermined time period, the reliability of detecting the light pointwithin the captured image data 23C captured within the predeterminedtime period is increased. It should be noted that the image capturedevice can be either an analog or digital image capture device and canbe either a still image capture device or a video device. The capturedimage data 23C is coupled to analyzer 25B that detects the laser pointdependent on the adjusted parameters.

[0022] It should be further noted that the laser point detector 25 canbe implemented by one of hardware, software, or firmware. It shouldfurther be noted that the laser point detector 25 can be embodied withinthe computer system 20 or external to the computer system 20 (as shownin FIG. 2).

[0023]FIG. 3 shows one embodiment of an apparatus for projecting a lightsignal 24A while transmitting, at essentially the same time, anotification signal 24B indicating that the light signal is beingprojected. As shown, the apparatus includes an activation option 30 thatwhen activated causes the emission of signals 24A and 24B. Theactivation option can comprise a button that when depressed, activatesthe apparatus or any alternative activation option. The apparatus can beembodied as a laser pointer having the additional feature of emitting anotification signal when the laser signal is being projected.Alternatively, the apparatus can be embodied as a computer mouse havingthe additional feature of emitting a laser signal at the same time asactivating the mouse. In this embodiment, the computer system is adaptedto receive the notification signal from the apparatus since there is apre-existing communication path between the computer mouse and thecomputer system.

[0024]FIG. 4 shows a first embodiment of the method of detecting a lightpoint on computer controlled displayed images. Initially, a light pointis projected on the displayed images at essentially the same time that anotification signal is transmitted indicating that the light signal isbeing projected (block 40). Next, at least one of image display or imagecapture parameters are adjusted to increase the detectability of thelight signal for a predetermined time interval (block 41) after receiptof the notification signal. For instance, image capture parameters maybe adjusted so as to cause a sensitivity to particular colors andintensities of the light point and/or to cause an insensitivity to allcolors and intensities that are not the same as or similar to the lightpoint. Alternatively, the display parameters can be adjusted such thatthe light point is more detectable within the adjusted displayed image(as will be described herein below).

[0025] The displayed image and the laser point are captured within thepredetermined time interval (block 42). The captured image data is thenanalyzed dependent on the adjusted parameters to detect the light pointwithin the displayed image (block 43). In particular, the captured datais analyzed to identify the pixels corresponding to the laser point anddetermine their corresponding location within the displayed image.

[0026] In one embodiment, prior to analyzing the captured data 23A todetect the laser point, captured data 23A is initially analyzed toidentify all pixel values that only correspond to the displayed imagewithin the captured data. An exemplary technique of locating the pixelvalues corresponding to the displayed image within the captured data isdescribed in U.S. application Ser. No. 09/774452 filed Jan. 30, 2001,entitled “A Method for Robust Determination of Visible Points of aControllable Display within a Camera View”, and assigned to the assigneeof the subject application (incorporated herein by reference). Ingeneral, the displayed image pixel values are located by initiallydisplaying and capturing a plurality of calibration images. Thecalibration images are single intensity greyscale images of a knownintensity. Constructive and destructive feedback data is derived fromthe captured calibration images by incrementing or decrementing pixelvalues in the captured data dependent on their intensity values. Thesub-set of pixels within the captured data having an intensity above agiven threshold after decrementing/incrementing correspond to thedisplayed image area within the captured image data.

[0027] Once, the sub-set of pixel corresponding to the displayed imageis determined, this sub-set of pixels can be analyzed, dependent on theadjusted parameters, to locate the pixel values corresponding to thelight point. For instance, if the adjusted parameters cause the lightpoint pixels to have a particular unique characteristic or cause thenon-light point pixels to have a particular unique characteristic, thenthe sub-set of pixels are analyzed to differentiate pixels dependent onthose unique characteristics.

[0028]FIG. 5 shows a second embodiment of the method of detecting alight point on a computer controlled displayed image. Similar to themethod as shown in FIG. 4, a light signal is projected at a computercontrolled displayed image while at essentially the same time anotification signal is transmitted indicating that the light signal isbeing projected (block 50). The displayed image intensity is reduced toat least a value that is less than a maximum value within apredetermined time interval after receipt of the notification signal(block 51). For example, referring to FIG. 2, the controller 25A cansend control to the computer system 20 to cause image data 20A to bedisplayed such that any pixel value does not exceed the maximum value.In general, it should be understood that the intensity of pixel valueshas a dynamic range as defined by [0, c₀]. For instance, an eight bitbinary pixel value has a dynamic intensity range of 0-255. Hence, in oneembodiment, the dynamic intensity range of the displayed image data isreduced by reducing pixel values of image data 20A to a value that isless than or equal to the maximum value. In one embodiment, the maximumvalue is half of the upper value (c₀) of the dynamic range of thedisplayed image (e.g., half of 255). In one embodiment, if pixel valuesdo not exceed the maximum value, they are not reduced. Alternatively,all pixel values are reduced by the same multiplicative factor such thatthe maximum pixel intensity is less than the maximum value.

[0029] After pixel values have been reduced, image data including thedisplayed image having reduced intensity and the projected light signalis captured within the predetermined time interval (block 52). The lightpoint is detected by identifying values within the captured image datathat are greater than the maximum value of the displayed image (block 53).

[0030] In one embodiment, pixel values can be reiteratively reduced,captured, and analyzed (blocks 51, 52, and 53) to detect the light pointas indicated by path 54 (FIG. 5). In this embodiment, the pixel valueintensity of the image data 20A is reduced as a function of time duringthe predetermined time interval until the light point is detected oruntil the end of the predetermined time interval. For instance, at thebeginning of the predetermined time interval (time t₀) the upper valueof the dynamic range c₀ of image data 20A is unchanged. At a time t₁within the predetermined time interval, a first maximum value isdetermined according to an intensity function at time t₁, and all pixelsin image data 20A having a value greater than the first maximum valueare reduced (block 51). Image data 23A is captured including theintensity reduced displayed images and the light point (block 52). Thesub-set of pixel values in the captured image data 23A corresponding tothe displayed image (as determined above) is analyzed and any pixelshaving an intensity greater than the first maximum value are identifiedas light point pixel values (block 53). If there are no pixel valuesgreater than the first maximum value, then at time t₂, the dynamic rangeof the displayed image data 20A is further reduced below a secondmaximum value as determined by the intensity function at time t₂ (block51 is repeated). Image data is again captured (block 52) and analyzed(block 53) to identify any of the sub-set of pixel values within thecaptured image data 23A corresponding to the displayed imaged that aregreater the second maximum value. This process is reiterated until theend of the predetermined time interval. The following is an exemplaryfunction for reducing pixel values: $\begin{matrix}{\quad {{{c_{\max}(t)} = {\max ( {0,{c_{o} \cdot ( \frac{T - t}{T} )}} )}},}} & {{Eq}.\quad 1}\end{matrix}$

[0031] where c_(max)(t) is the maximum value as a function of time t, c₀is the original upper intensity value of the dynamic range of thedisplay (e.g., 255), and T is the length of the predetermined timeinterval. It should be noted that as t approaches T, c_(max)(t)approaches zero. It should be further noted that any function thatapproaches zero as t approaches T can also be employed to set themaximum values.

[0032] Once the light point is detected within the captured image data23C, its coordinate location within the captured image data is alsoknown. This coordinate location can be translated into a coordinatelocation within the displayed image data 20A. In an exemplary techniquefor determining the coordinate location of the light point within theimage data 20A given its coordinate location within the captured data, amapping function is derived between the coordinate system of the set ofpixels values within the captured image data 23A corresponding to thedisplayed image and the coordinate system of the displayed image data20A. In one embodiment, this mapping function is derived as described inU.S. application Ser. No. 09/775032 filed Jan. 31, 2001, entitled “ASystem and Method For Robust Foreground And Background Image DataSeparation For Location Of Objects In Front Of A Controllable DisplayWithin A Camera View”, and assigned to the assignee of the subjectapplication (incorporated herein by reference).

[0033] By determining the corresponding location of the light pointwithin the image data 20A, the light point user can control the displaysystem dependent on the placement of the light point within thedisplayed image. For instance, by placing the light point in aparticular position on the displayed image, the user can providecontrol, for instance, to the operating system or applications runningon the computer system. In addition, the information can be used tocause the application software generating the slide images to switch toa new slide. In general the information can be converted to correspondto any input signal that the computer system might expect to receivefrom a conventional input device such as a keyboard or a mouse. Inanother embodiment, the information can be provided to the displaydevice to cause it to adjust its settings. In still another embodiment,the information can be provided to the image capture device forcontrolling the parameters (e.g., resolution) of capturing image data.

[0034] Hence, a system, method, and apparatus are described fordetecting a light point projected on computer controlled displayedimages that allows a user to easily interact with the display systemusing a single device.

[0035] In the preceding description, numerous specific details are setforth in order to provide a thorough understanding of the presentinvention. It will be apparent, however, to one skilled in the art thatthese specific details need not be employed to practice the presentinvention. In addition, it is to be understood that the particularembodiments shown and described by way of illustration is in no wayintended to be considered limiting. Reference to the details of theseembodiments is not intended to limit the scope of the claims.

I claim:
 1. A system including a computer system for controlling andgenerating image data for displaying an image, the system comprising:device for projecting a light point at the displayed image while atessentially the same time transmitting a notification signal indicatingthe light point is being projected; controller for adjusting at leastone of image capture and image display parameters so as to increasedetectability of the light point for a predetermined interval of timeafter receipt of the notification signal; device for capturing imagedata including the displayed image and the projected light point withinthe predetermined time interval; analyzer for detecting the light pointwithin the captured image data dependent on the adjusted parameters. 2.The system as described in claim 1 wherein the controller is an imagedisplay controller for reducing displayed image intensity for thepredetermined time interval to at least an intensity value that is lessthan or equal to a maximum value.
 3. The system as described in claim 2wherein the image analyzer detects the light point within the capturedimage data by identifying intensity values greater than the maximumvalue.
 4. The system as described in claim 2 wherein the image displaycontroller reduces displayed image intensity as a function of time forthe predetermined time interval until the light point is detected oruntil the time interval has ended.
 5. The system as described in claim 1wherein the device for projecting a light point is a laser pointer. 6.The system as described in claim 1 wherein the light point is projectedand the notification signal is transmitted upon activation of a singlecontrol option on the light projection device.
 7. The system asdescribed in claim 1 wherein the notification signal is transmitted byone of a wireless and wired connection.
 8. The system as described inclaim 1 wherein the notification signal is transmitted via an infraredsignal.
 9. The system as described in claim 1 wherein the image analyzeris implemented by one of hardware, software, and firmware.
 10. Anapparatus where upon activation projects a visible light point while atessentially the same time transmits a notification signal indicating thelight point is being projected.
 11. The apparatus as described in claim10 wherein the notification signal is used to control a display systemto adjust at least one of image capture and image display to increasethe detectability of the light point within a predetermined interval oftime after receipt of the notification.
 12. The apparatus as describedin claim 11 wherein the light point is a laser signal.
 13. The apparatusas described in claim 11 wherein the light point is projected and thenotification signal is transmitted upon activation of a single controloption on the apparatus.
 14. The apparatus as described in claim 11wherein the notification signal is transmitted by one of a wireless anda wired connection.
 15. The apparatus as described in claim 11 whereinthe notification signal is transmitted via an infrared signal.
 16. Theapparatus as described in claim 11 wherein it is a handheld device. 17.The apparatus as described in claim 11 wherein it is one of a computermouse and a laser light pointer.
 18. An apparatus comprising: means forprojecting a visible light point on a computer controlled displayedimage; and means transmitting a notification signal at essentially thesame time as the visible light point, the notification signal indicatingthe light point is being projected.
 19. The apparatus as described inclaim 18 wherein the notification signal is used to control a displaysystem to adjust at least one of image capture and image display toincrease the detectability of the light point within a predeterminedinterval of time after receipt of the notification signal.
 20. Theapparatus as described in claim 19 wherein the light point is a lasersignal.
 21. The apparatus as described in claim 19 wherein the lightpoint is projected and the notification signal is transmitted uponactivation of a single control option on the apparatus.
 22. Theapparatus as described in claim 19 wherein the notification signal istransmitted wirelessly.
 23. The apparatus as described in claim 19wherein the notification signal is transmitted over a wired connection.24. The apparatus as described in claim 19 wherein the notificationsignal is transmitted via an infrared signal.
 25. The apparatus asdescribed in claim 19 wherein it is a handheld device.
 26. The apparatusas described in claim 19 wherein it is one of a computer mouse and alaser light pointer.
 27. A method for use in a display system includinga computer system for generating image data for displaying an image, themethod comprising: projecting a light point at the displayed image whileat essentially the same time transmitting a notification signalindicating the light point is being projected; adjusting at least one ofimage display and image capture parameters so as to increasedetectability of the light point for a predetermined interval of timeafter receipt of the notification signal; capturing image data withinthe predetermined time interval including the displayed image and theprojected light point; analyzing the captured image data to detect thelight point within the displayed image dependent on the adjustedparameters.
 28. The method as described in claim 27 wherein adjusting atleast one of image display and image capture comprises reducing thedisplayed image intensity to at least an intensity value that is lessthan a maximum value.
 29. The method as described in claim 27 whereinadjusting at least one of image display and image capture comprisesreducing the dynamic intensity range of the displayed image data. 30.The method as described in claim 28 wherein analyzing image data furthercomprises identifying intensity values greater than the maximum value inthe captured image data to detect the light point within the displayedimage.
 31. The method as described in claim 27 wherein adjusting atleast one of image display and image capture comprises increasingresolution when capturing image data.
 32. The method as described inclaim 27 wherein adjusting at least one of image display and imagecapture comprises increasing sensitivity to a particular color andintensity associated with the light point when capturing image data. 33.The method as described in claim 27 where adjusting at least one ofimage display and image capture comprises decreasing sensitivity tocolors other than a particular color and intensity associated with thelight point when capturing image data.
 34. The method as described inclaim 32 where adjusting at least one of image display and image capturecomprises decreasing sensitivity to colors other than a particular colorand intensity associated with the light point when capturing image data.35. The method as described in claim 27 further comprising: adjusting atleast one of image display and image capture parameters by reiterativelyreducing as a function of time the displayed image intensity during thepredetermined time interval; and capturing image data within thepredetermined time interval by capturing the displayed image and thelight point at least each time the displayed image intensity is reduced;and identifying intensity values greater than a maximum value in thecaptured image data to detect the light point within the displayedimage.